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sion to the controller computer where the commands 
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Description 

The present invention relates to remote control systems a* particuiarly to a system and method for dup.icating the 
soreen image of a source data processing system on a ^"^^Ss^ey go beyond just connecting a 
Bemotecontrol programs are a S P^^ C ^ modems or 

computer wHh another computer to actu^lytakmg ^^J^S!2!^SJln «y*« (e.g. a persona, 
through LAN connections, these Programs can put all ^ r « s ° ur ^^ Lto control of another data processing system 
computer or a workstation), usually caHed ^^aKSyS ^roller. The controHer screen 
(e.g. a personal computer or a wortet ^° n ; ° onn ^ 
,o duplicatesthetargetone.Theke^ 

tern; the mouse of the controller can ^ «2 Soard of the controHer sends data to the 

^pTeS 

sonal confer system cn be controlled and g«MM . to* ^ Brgel nc can be controlled 

boost performances, remote control programs try to r ^^"^^ JTISeo imaoe on the display screen of 
ted over the communication lines W~*g™>^^ the fransm P jssion spee d. 

the target PC. which should be °"J*j£d EEJX remote control programs worked by send- 

due to the big quantity of informal ^^£™£^£SZk every fraction of a second. The constant 

to be transmitted at each refresh. ,^« rtr , „, tho OTP o„ / e Q movina the mouse pointer, pulling down 

Often, a change of the image is limrted to a small MM of the JJJJJ^^ 7 ystem can use the 

a menu, highlighting an object) and only the changed data b * ^ een By cutting out redundant screen 

screen data previously received and just refresh ^^^^^^miSi speed the performance of 
refresh data, a remote control program can .mprove the response ^<**»™2r^ a ^ is to keep 
hard drrves and memory chips, and can also , be useful ^X^^^L soon needed again. ■ is much 
recently used data in a temporary storage area that is quickly acces suae n mo «. According 
quicker to access them from a fast-access ^temporary *J^5^3S^ The entire page 

to this technique, elements of a recently drspl «J SiSS l^rge amounts of information over the 

srr r tSr ^=ric h ^cr = ss 12 s -J- frOT . cace « 
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maps, text and fonts. ♦rancmiccion of bit-maD imaqes, by limiting the trans- 

US patent 5.491 .780 describes a technique tor reducng «£*^X^nta wmmaSs of the target computer, 
mission and the refreshing only to the portion of screen >«**£Mdtay ^^J^SSthe communication lines. 

rKSKSSSS.-* ^r^ooo^eoan^^tnoae. 
processing system comprising: 
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- a first display unit for displaying graphic images and 

- a first display driver for modifying the image on the first display unit. 

the source data processing system comprising: 

- a second display unit for displaying images, 
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erated by the graphic engine; 

5 the at .east one destination data procesang system and the source data processing system being connected through 
a network, the method comprising the steps of: 

- intercepting the graphic commands generated by the graphic engine; 

- reSingThrsystem independent graphic commands, through the network, to the at .east one destination data 

- ^m^ystem independent graphic commands into the at .east one destination data processing system 

15 format, according to said protocol; 

- passing the transformed graphic commands to the first d IS play dnver. 

20 destination data processing system comprising . 

- a f irst display unit for displaying graphic images and 

- a f irst display driver for modifying the image on the f irst display unit; 

25 the source data processing system comprising: 

- a second display unit for displaying images. 

30 erated by the graphic engine; 

the at .east one destination data processing system and the source data processing system being connected through 
a network, the system comprising: 

means for intercepting the graphic commands generated by the 

. ranstrr^ 

. ^mea^s^elng the system independent graphic commands to the at .east one destination data 

- SSSrmingtn^ 
ing system format, according to said protocol; rtisolav driver 

- communication means for passing the transformed graph,c commands to the first d.splay dnver. 

Various embodiments of the invention wi.lnow bedescribed in detai. by way of examp.es. with reference to accom- 
panying figures, where: 

Fig. 1 showsschematica.lythenetworkandthepe^ 
and system of the present invention; 

Fig. 4 shows schematically an embodiment of the present invention. 
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The data processing system 105 is the controller and can aKernatively take the control of one of the other data 
processing system 103, 107 and 109. computers or workstations available on the mar- 

is lic^^s^^™^ * w* «-*• **» 

10 S™^"^ pS^S^ed above tnay be „*emer«ec ■ 

en, s^«eX^aT^Tsl* as .heW Persons! Coe^e, .danced by ,«e,oa.«x, a Bosmess 

any f* 31 ^ , pcy „ ta a fr ademark of international Business Machines Corporation) 

^££T££^U-Y Par, .) «. operaw system .Ka. iedudes e co,,ec.» * rf* p~ 

readies oi system ,03 cen be used and actveted by the conwlle, syste^Fo, perform* suet, bomroi. system ,05 
™stbeabtetorepteser«or.itsdlsplayaneiiacteopyol»,etaroetdisplay103. ,„„.„„^ m .. rt ™ oh i ca | 

Obviously, this solution is much more efficient in terms of speed of transmission than the method of sending a M 

^iSSirSJ J.e e bit-map is a sort of "universal format" which can be understood and processed by any operating 
systen^^^^ commands can create some prcblems in terms of compatibility n the operating sys- 

^ VSSX^^^^S^ (209) according to the present invention, translates the graphic 
command iiued bXgraphic engine (203) of the target computer into a system independent format us.ng a prede- 
«nX^Ti?e^ M^Lt commands are then transmitted through the netwo rk to the controller compu- 
tlTiSSw SJ anslated in graphic commands for the display driver (213) using the same protocol. 

' 1^*Z* Tpr«J* embodiment of the present invention, only a subset of the ^<^**TS2 
the graphic engine are translated and transmitted to the controller. The reason « that a hmrted set of commands covers 
so the most of the operations performed by a display driver. 

The following is a definition of the constants and data structures expressed ,n the C language as used .n a pre 
ferred embodiment of the present invention: 
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List of constants and data structure expressed in X? language used in the protocol: 
Command identifier values: 

#define ACCM__CMD__B I TBLT 1 

5 #define ACCH_CMD_JBITBLT_£AVE 2 

#define ACCM_CMD_B I TBLT_H I T 3 

#define ACCM_CMD_BITBLT_SRCCOPY 4 

#define ACCM_CMD_SAVESCREENBi™AP_SAVE 5 

#define ACCM_CMD_SAVESCREENBITMAP_RESTORE 6 

#define ACCM_CMD_B ITBLT_REPEAT 7 

10 #define ACCM_CMD_TEXTOUT 8 

Brush field values: 

#define DIC_BRUSH_SOLID 0 

#define DIC_BRUSH_HATCHED 2 

15 #define DIC_BRUSHJPATTERN 3 

#define D I C_BRUSH_SIMUIATED_ PATTERN 4 

Extended Text Out field values: 

#define ETO_OPAQUE 0x0002 
#define ETQ_CLIPPED 0x0004 
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Background field values: 

#define BK_TRANS PARENT 
#define BK_OPAQUE 

Font identifier field values: 

#de f ine F0NT_C0URIER 

#define F0NT_HELV 

tdefine FONT_S Y STEM 

#de f ine FONT_TMS ROMAN 



typedef struct tagDCAF^ BITWAPINFOHEADER 

{ _ 

unsigned short usSize; 

unsigned short usBmpWidth; 

uns igned short usBmpHeight; 

unsigned char bPlanes; 

unsigned char bBitcount; 

ty ie^SS— .POC^BI^PXNFOHEADEH 
typedef struct tagDCAF^BITMAPINFO 

tyy£f^SS53Si»0 .PDCAF.BI^PINFO; 

typedef struct tagDCAF_BRUSH INFOHEADER 

( 

short sSize; 

short sType ; 

long IColor; 

short sHatch; 

long lBkColor ; 

short usXal ign ; 

short usYalign; 

tyi-SfSSSSSSSSS^l * PDCAF_BRUSH INFOHEADER ; 

typedef struct tagDCAF_BRUSHINFO 

* DC AF_BRUSH INFOHEADER dcf brushinf oheader ; 

DCAF_BITMAPINFO dcfbmpinfo; 
) DC AF_BRUSH INFO ; 
typedef DC AF_B RUSH INFO *PDCAF_BRUSHINFO; 

typedef struct tagDCAF_BITMAPSAVER 

( 

unsigned short usBmpSize; 
unsigned short usWidth; 
unsigned short usHeight; 
unsigned short usWidthBytes ; 
unsigned short usPlanes; 
char bmp; 
) DCAF_BITMAPS AVER ; 
typedef DC AF_B ITMAPS AVER * PDCAF_BITMAPSAVER ; 

typedef struct _DCAFR£CT { 

unsigned short usRectXLeft; 

unsigned short usRectYBottom; 

unsigned short usRectXRight ; 

unsigned short usRectYTop; 

} DCAFRECT; 
typedef DCAFRECT *P DCAFRECT; 
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typedef struct _ACCM_TEXTOUT ( 
unsigned char bCoramandType ; 
unsigned short usCmdLength; 
unsigned short usEtoOpt ions ; 
char bClipPresent; 
DCAFRECT rectClipArea; 
char bOpaque Present ; 

DCAFRECT rectOpaqueArea ; 
unsigned short usDestX; 
unsigned short usDestY; 
unsigned short usCount; 
unsigned short usSelHeight; 
unsigned short usSelWidth; 
unsigned short usPointSize; 
unsigned short usFaceNaroe; 

unsigned long ulBkColor; 

unsigned long ulTextColor; 

unsigned short usBkMode; 

unsigned short usWeight; 

unsigned short usWidth; 

unsigned short usHeight; 

char cltalic; 

char cUnderline; 

char cStrikeOut; 

char cText ; 



) ACCM^TEXTOUT; 
typedef ACCM_TEXTOUT 



' PACCM_TEXTOUT ; 



typedef struct __ACCM_BITBLT_CORE { 

unsigned char bCommandType ; 

unsigned short usCrodLength; 

unsigned short usDestXOrigin; 

unsigned short usDestYOrigin; 

unsigned short usSrcXOrigin; 

unsigned short usSrcYOrigin ; 

unsigned short usXExtension; 

unsigned short usYExtension; 

long lRop3 ; 

) ACCM_BITBLT_CORE; 
typedef ACCM_B ITBLT_CORE *PACCM_BITBLT_CORE; 

typedef struct _ACCM_BITBLT { 

ACCM_B ITBLT_CORE bmp_core ; 
DCAF_BRUSHINFO Brush; 

) ACCM__B I TBLT ; 
typedef ACC14_BITBLT * PACCM_ BITBLT ; 



typedef struct _ACCM_B ITBLT_S AVE { 

ACCM_BITBLT_CORE bmp_core; 

unsigned char bBmpID; 

DC AF_B ITMAPS AVER bmpSaver; 

) ACCM_BITBLT_SAVE; 
typedef ACCM_BITBLT_SAVE *PACCM_BITBLT_SAVE; 

typedef struct _ACCM_B I TBLT_H I T { 

AC CM_B I TB LT_C O RE bmp_core ; 

unsigned char bBmpID; 

) ACCM_BITBLT_HIT; 
typedef ACCM_BITBLT_HIT * PACCM_J3ITBLT_HIT ; 

typedef struct _ AC CM_S AVES C REENB I TMAP ( 
unsigned char bCommandType ; 
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unsigned short usCmdLength ; 
unsigned short usRectXLeft; 
unsigned short usRectYBottora; 
unsigned short usRectXRight ; 
unsigned short usReetYTop; 

) AC CM_S AVES C REENB I TMAP ; __ T _ T ^._ 

typedef ACCM_SAVESCREENBITMAP •PACCM.SAVESCREENBI'IMAP; 



The information necessary to replay a graphical command are stored in a structure which contains the command 
identifier and the parameters used to replay the command. For example when a text is drawn the information regarding 
this event is stored in the structure ACCM_TEXTOUT whose first field contains the value ACCM_CMD_TEXTOUT and 
the other fields are used to replay the text. 

The protocol described above is believed to handle most of the events that modify the display on any known oper- 
ating system. For example, using the Microsoft Windows operating system the above described protocol is able to hook 
and replay an average of 95% of the screen changes. 

Fig 3 shows the steps of the method according to a preferred embodiment of the present invention. At step 301 the 
server 101 connects the target computer 1 03 with the controller computer 105. At step 303 the hook display driver 209 
intercepts the graphic commands issued by the graphic engine 203 to the display driver 205. Step 305 identifies those 
graphic commands belonging to a predefined subset of commands and extracts the related parameters. Step 307 
translates the graphic commands identified by step 305 and the related parameters into a system independent format 
according to the predefined protocol described above. Step 309 sends the translated commands and parameters to the 
display driver 213 of the controller computer, where the commands will be retranslated from the system independent 
format into the controller system format to be reproduced on the controller display 215. 

Those skilled in the art will appreciate that a proper translation tailored on the format of each system (source or 
controller) must be implemented in order to realize the correspondence between two different systems and the common 
protocol However, due to the limited number of commands belonging to the subset, this additional operations do not 
constitute a substantial overhead neither in the functioning of the system nor from the programming point of view. 

According to a preferred embodiment of the present invention the complete correspondence of the controller dis- 
play with the target display can be granted by sending every predetermined period of time (e g. 5 sec.) a bit-map of the 
image actually displayed on the target display to "synchronize" the controller display in case of events not covered by 

the protocol. „ , 

To check whether the image replayed on the controller display is coincident with the target display, a shadow bit- 
map copy of the target display is maintained in the storage of the target computer. Each time a command issued by the 
graphic engine is translated using the protocol and transmitted to the controller computer to be re-translated and 
replayed the same operation is done on the "shadow" bit-map image: the system independent commands are re-trans- 
lated and operated on the "shadow" bit-map. Every predetermined period ot time (5 sec. in the preferred embodiment) 
the "shadow" image is compared with the actual target display and if any difference is detected, the portion of the 
screen to be modified is transmitted in the bit-map format to the controller computer. 

Fig 4 shows schematically the "synchronization" mechanism: a subset of the graphic commands, issued by the 
graphic engine 203 to the display 207 in the target PC. is intercepted and translated by translator 403 and sent to the 
controller PC where they will be re-translated and reproduced on display 215. This last operation is simulated also on 
the "shadow bit-map" 401 : in this way the target PC can always check what is actually displayed on controller display 
215. Every predetermined period of time (e.g. 5 seconds) the shadow bit-map is compared with the bit-map of the dis- 
play 207 and where a difference is detected a "synchronization" is performed: the bit-map is transmitted to the controller 
display 215 to update the displayed image and re-establish the conespondence between the two images. 

The bit-map transmission, as mentioned above, is a heavy and time consuming activity, but according to the syn- 
chronization mechanism described above it is performed only once every period of time and only when actually needed. 
In this way the high reliability of this kind of transmission is obtained while not affecting the speed of the communications 
between the systems. 

Claims 

1 . A method for duplicating and refreshing, on at least one destination data processing system a graphic image dis- 
played on a source data processing system, the at least one destination data processing system comprising: 

a first display unit tor displaying graphic images and 
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- a first display driver for modifying the image on the first display unit; 
the source data processing system comprising: 

- a second display unit for displaying images, 

generated by the graphic engine; 
the at least one destination data processing system and the source data processing system being connected 
through a network, the method comprising the steps of: 

- intercepting the graphic commands generated by the graphic engine; 

- passing the graphic command to the second display driver; onmHinBtoa 

- ESSng a subset of the graphic commands into an operating system independent format. accord,ng to a 

- iX^sySmindependent graphic commands, through the network to the at least one destination data 

- Sa^rmTngTsystem independent graphic commands into the at least one destination data processing sys- 
tern format, according to said protocol; 

- passing the transformed graphic commands to the first display driver. 

2. The method of claim 1 further comprising the steps of: 

- copying in the storage of the source data processing system a bit-map of the image displayed on the second 

display unit; . . . 

- modifying the bit-map image according to said subset of graphic commands. 

- comparing, every predetermined period of time, the bit-map image with the image displayed on the second d,s- 

Xe 'difference between the bit-map image and the image displayed on the second display unit exceeds a pre- 
ZSSl, updating the bit-map image and sending, through the network, the bit-map image to the 
at least one destination data processing system. 

3. The method of any preceding claim wherein the network is a Local Area Network. 

4. A remote control system using the method of any preceding claim. 

5 A system for duplicating and refreshing, on at least one destination data processing system a graphic image dis- 
p.ay^ on a source data processing system, the at least one destination data processmg system composing . 

a first display unit for displaying graphic images and 

- a first display driver for modifying the image on the first display unit; 

the source data processing system comprising: 

- a second display unit for displaying images, 

generated by the graphic engine; 
the at least one destination data processing system and the source data processing system being connected 
through a network, the system comprising: 

- means for intercepting the graphic commands generated by the graphic engine. 

- communication means for passing the graphic command to the second display d"ver 

- means for translating a subset of the graphic commands into an operating system .ndependent format, accord 

- nX^tSSthe system independent graphic commands to the at .east one destination data 
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processing system; 

- means for transforming the system independent graphic commands into the at least one destination data 
processing system format, according to said protocol; 

- communication means for passing the transformed graphic commands to the first display driver. 

5 

6. The system of claim 5 further comprising: 

- timer means tor counting a predetermined period of time; H icni*vpH nn the 

- means for copying in the storage of the source data processing system a bit-map of the image displayed on the 

10 second display unit; 

- means for modifying the bit-map image according to said subs . . 

- means for comparing, every predetermined period of time, the bit-map image with the image ^^donjhe 
second display unit and tor updating the bit-map image if the difference exceeds a predetermined threshold 

- network means for sending the bit-map image to the at least one destination data processing system, if the drf- 
, 5 ference between the bit-map image and the image displayed on the second display unit exceeds a predeter- 
mined threshold 

7. The system of claim 5 or 6 wherein the network is a Local Area Network 
20 8. A remote control system implementing the system of claim 5, 6 or 7. 
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target computer by a controller computer through a net- 
work. The graphic commands issued by the graphic 
engine of the target computer are "hooked" and trans- 
lated into a system independent format. The translated 
commands are then transmitted to the controller com- 
puter to be replayed on the display. The independency 
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sion to the controller computer where the commands 
are re-translated according to the same protocol. 
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